Image classification using CNNs in Keras

Context:

Can you differentiate a weed from a crop seedling?

The ability to do so effectively can mean better crop yields and better stewardship of the environment.

The Aarhus University Signal Processing group, in collaboration with University of Southern Denmark, has recently released a dataset containing images of unique plants belonging to 12 species at several growth stages.

Learning Outcomes:

• Pre-processing of image data.
• Visualization of images.
• Building CNN.
• Evaluate the Model.
• The motive of the project is to make the learners capable to handle images/image classification problems, during this process you should also be capable to handle real image files, not just limited to a numpy array of image pixels.

Guide to solve the project seamlessly:

Here are the points which will help you to solve the problem efficiently:

• Don't use data from Kaggle. Instead use the data provided with this problem statement.
• Upload the "images.npy" and “Labels.csv” file to google drive.
• Then you can use the dataset path in the Google Colab notebook to do further steps related to project problem statement.
• You can set runtime type to “GPU” in Google Colab, so that the code will run faster as you will be using CNN to fit your model.

Data Description:

You are provided with a dataset of images of plant seedlings at various stages of grown. Each image has a filename that is its unique id. The dataset comprises 12 plant species. The goal of the project is to create a classifier capable of determining a plant's species from a photo.

Dataset:

The data file names are:

• images.npy
• Label.csv

The original files are from Kaggle. Due to the large volume of data, the images were converted to images.npy file and the labels are also put into the Labels.csv. So that you can work on the data/project seamlessly without worrying about the high data volume. The following code was used to convert the large dataset of images to numpy array:

The data before conversion was downloaded from Kaggle project site: https://www.kaggle.com/c/plant-seedlings-classification/data?select=train

#This code is an illustration of how the data for this project was generated.

# Import necessary libraries.

#import math
#import numpy as np
#import pandas as pd
#from glob import glob

#data_path = '/content/drive/My Drive/Colab Notebooks/data/plant_seedlings/train.zip'
#!mkdir dataset

# Extract the files from dataset to temp_train and temp_test folders (as the dataset is a zip file.)
#from zipfile import ZipFile
#with ZipFile(data_path, 'r') as zip:
# zip.extractall('./dataset')
#path = "/content/dataset/*/*.*" # The path to all images in training set. (* means include all folders and files.)

#files = glob(path)
#trainImg = [] # Initialize empty list to store the image data as numbers.
#trainLabel = [] # Initialize empty list to store the labels of images
#j = 1
#num = len(files)
# Obtain images and resizing, obtain labels
#for img in files:
# Append the image data to trainImg list.
# Append the labels to trainLabel list.
# print(str(j) + "/" + str(num), end="\r")
# trainImg.append(cv2.resize(cv2.imread(img), (128, 128))) # Get image (with resizing to 128x128)
# trainLabel.append(img.split('/')[-2]) # Get image label (folder name contains the class to which the image belong)
# j += 1
#trainImg = np.asarray(trainImg) # Train images set
#trainLabel = pd.DataFrame(trainLabel, columns=["Label"]) # Train labels set
#print(trainImg.shape)
#print(trainLabel.shape)
#trainLabel.to_csv('Labels.csv', index=False)
#np.save('plantimages', trainImg)

Steps and tasks:

Deliverable – 1:

Import the libraries, load dataset, print shape of data, visualize the images in dataset. (5 Marks)

Deliverable – 2:

Data Pre-processing: (15 Marks)

1. Normalization.
2. Gaussian Blurring.
3. Visualize data after pre-processing.

Deliverable – 3:

Make data compatible: (10 Marks)

1. Convert labels to one-hot-vectors.
2. Print the label for y_train[0].
3. Split the dataset into training, testing, and validation set. (Hint: First split images and labels into training and testing set with test_size = 0.3. Then further split test data into test and validation set with test_size = 0.5)
4. Check the shape of data, Reshape data into shapes compatible with Keras models if it’s not already. If it’s already in the compatible shape, then comment in the notebook that it’s already in compatible shape.

Deliverable – 4:

Building CNN (15 Marks)

1. Define layers.
2. Set optimizer and loss function. (Use Adam optimizer and categorical crossentropy.)

Deliverable – 5:

Fit and evaluate model and print confusion matrix. (10 Marks)

Deliverable – 6:

Visualize predictions for x_test[2], x_test[3], x_test[33], x_test[36], x_test[59]. (5 Marks)

Note:

Do not download the dataset from Kaggle, as:

• The dataset is big.
• The dataset has 2 files for train and test images, but the labels are only for the train file. Test file has no labels associated with it. So, when you want to know the accuracy of model on test images, there’s no way to measure it. That’s why the data provided has only train images and their labels. For our purpose we use this for our training and testing and validation purpose.

Index:

• **Deliverable – 1**
• **Deliverable – 2**
• **Deliverable – 3**
• **Deliverable – 4**
• **Deliverable – 5**
• **Deliverable – 6**

Deliverable – 1:

Back to Index

from google.colab import drive
#drive.mount('/content/drive')

%tensorflow_version 2.x
import tensorflow
tensorflow.__version__

'2.4.1'

#import libraries
import numpy as np
import pandas as pd
import warnings
warnings.filterwarnings("ignore")

import matplotlib.pyplot as plt
import seaborn as sns
import matplotlib.cm as cm
%matplotlib inline 

from sklearn import metrics
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Conv2D, Dropout, MaxPooling2D, Flatten
from keras import models, layers, callbacks

from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import train_test_split

from sklearn.metrics import confusion_matrix, classification_report

import cv2
from google.colab.patches import cv2_imshow

from sklearn.preprocessing import LabelEncoder
from tensorflow.keras.utils import to_categorical

#Read Data
from google.colab import drive
drive.mount('/gdrive')
%cd /gdrive/'My Drive'/'Colab Notebooks'
labels  = pd.read_csv('Labels.csv')
labels.head(5)

Drive already mounted at /gdrive; to attempt to forcibly remount, call drive.mount("/gdrive", force_remount=True).
/gdrive/My Drive/Colab Notebooks

	Label
0	Small-flowered Cranesbill
1	Small-flowered Cranesbill
2	Small-flowered Cranesbill
3	Small-flowered Cranesbill
4	Small-flowered Cranesbill

images = np.load('images.npy')

#Stastical Analysis
print(images.shape)

print(labels.shape)

(4750, 128, 128, 3)
(4750, 1)

# Following code gives the first image. The code is commented for better visibility of notebook.

#images[0]

# Following gives the first row in the first image. The code is commented for better visibility of notebook.

#images[0][0]

#Following gives the first pixel in the first image.

images[0][0][0]

array([35, 52, 78], dtype=uint8)

#Step4: Find number of unique values in each column. Analyse outcome for categorical variables.
labels.nunique() # Number of unique values in a column

Label    12
dtype: int64

# value counts gives us how many times does the value 
print(labels['Label'].value_counts(normalize=True))

Loose Silky-bent             0.137684
Common Chickweed             0.128632
Scentless Mayweed            0.108632
Small-flowered Cranesbill    0.104421
Fat Hen                      0.100000
Charlock                     0.082105
Sugar beet                   0.081053
Cleavers                     0.060421
Black-grass                  0.055368
Shepherds Purse              0.048632
Common wheat                 0.046526
Maize                        0.046526
Name: Label, dtype: float64

print(labels['Label'].value_counts())

Loose Silky-bent             654
Common Chickweed             611
Scentless Mayweed            516
Small-flowered Cranesbill    496
Fat Hen                      475
Charlock                     390
Sugar beet                   385
Cleavers                     287
Black-grass                  263
Shepherds Purse              231
Common wheat                 221
Maize                        221
Name: Label, dtype: int64

sns.countplot(y=labels["Label"])

<matplotlib.axes._subplots.AxesSubplot at 0x7fe5acc984d0>

Observation:

Back to Index

1. Some classes e.g. Loose Silky-bent are represented more frequently in dataset than some other classes e.g. Maize, Common Wheat etc. Therefore some bias and classification error might occur.

from random import randint


from keras.preprocessing import image

# Find the start and end indexes of each classification
fig= plt.figure(figsize= (18, 40))
fig.suptitle('Random Samples From Each Class', fontsize=14, y=.92, horizontalalignment='center', weight='bold')

columns = 5
rows = 13

labelss = list(labels.Label.unique())

for i in range(12):
  lab = labelss[i]
  a = list(labels[labels['Label']==lab].index.values)
  fst_index = a[0]
  lst_index = a[-1]
  for j in range(1,6):
    fig.add_subplot(rows, columns, i*5+j)
    plt.axis('off')
    if j==1:
      plt.text(0.0, 0.5,lab, fontsize=15, wrap=True)
      continue
    r = randint(fst_index, lst_index)
    random_image= images[r]
    plt.imshow(random_image)
    
plt.show()

Observation:

Back to Index

1. There are 4750 R,G,B images (Channel = 3) with 128 X 128 size.
2. Each image contains 128 rows of 128 pixels. Each pixel is represented as an array of RGB values. e.g. The first pixel (top-left) of the first image has RGB values as [ 35, 52, 78].
3. The number of labels in csv file are same as number of images in dataset. No discrepancies.
4. Frequency analysis shows that there are more images of unwanted crops than wanted ones.
5. Some images were printed. It could be observed that the images contain leaf parts only.
6. Only a small part of the image is (e.g. leaf) is the data of importance. Remaining all data from the image (e.g. background) consists of noise.

Deliverable – 2:

Back to Index

Approach:

Back to Index

Data Pre-processing is performed as follows:

1. Normalization.
2. Gaussian Blurring.
3. Visualize data after pre-processing.

# 1. Normalization: 
# First convert the datatype of each RGB value in each pixel to float. Currently its a whole number 
# ranging between 0 - 255
images = np.load('images.npy')

images1 = images.astype('float32')

# For normalization, divide each value by 255
images1 /= 255

images1[3731][0][0]

array([0.19607843, 0.2901961 , 0.3882353 ], dtype=float32)

# Visualize images after normalization

fig= plt.figure(figsize= (18, 40))
fig.suptitle('Random Samples From Each Class', fontsize=14, y=.92, horizontalalignment='center', weight='bold')

columns = 5
rows = 13

labelss = list(labels.Label.unique())

for i in range(12):
  lab = labelss[i]
  a = list(labels[labels['Label']==lab].index.values)
  fst_index = a[0]
  lst_index = a[-1]
  for j in range(1,6):
    fig.add_subplot(rows, columns, i*5+j)
    plt.axis('off')
    if j==1:
      plt.text(0.0, 0.5,lab, fontsize=15, wrap=True)
      continue
    r = randint(fst_index, lst_index)
    random_image= images1[r]
    plt.imshow(random_image)
    
plt.show()

Observation:

Back to Index

No change in image even after normalization.

#Check how the images blur when different filters are applied

blur_img = []
for i in images1:
  blur_img.append(cv2.GaussianBlur(i, (15, 15), 0))

# Visualize Images After Guassian Blurring

fig= plt.figure(figsize= (18, 40))
fig.suptitle('Random Samples From Each Class', fontsize=14, y=.92, horizontalalignment='center', weight='bold')

columns = 5
rows = 13

labelss = list(labels.Label.unique())

for i in range(12):
  lab = labelss[i]
  a = list(labels[labels['Label']==lab].index.values)
  fst_index = a[0]
  lst_index = a[-1]
  for j in range(1,6):
    fig.add_subplot(rows, columns, i*5+j)
    plt.axis('off')
    if j==1:
      plt.text(0.0, 0.5,lab, fontsize=15, wrap=True)
      continue
    r = randint(fst_index, lst_index)
    random_image= blur_img[r]
    plt.imshow(random_image)
    
plt.show()

blur_img = []
for i in images1:
  blur_img.append(cv2.GaussianBlur(i, (5, 5), 0))

plt.imshow(blur_img[3731])

<matplotlib.image.AxesImage at 0x7fe5a0189b10>

# Visualize Images After Guassian Blurring

fig= plt.figure(figsize= (18, 40))
fig.suptitle('Random Samples From Each Class', fontsize=14, y=.92, horizontalalignment='center', weight='bold')

columns = 5
rows = 13

labelss = list(labels.Label.unique())

for i in range(12):
  lab = labelss[i]
  a = list(labels[labels['Label']==lab].index.values)
  fst_index = a[0]
  lst_index = a[-1]
  for j in range(1,6):
    fig.add_subplot(rows, columns, i*5+j)
    plt.axis('off')
    if j==1:
      plt.text(0.0, 0.5,lab, fontsize=15, wrap=True)
      continue
    r = randint(fst_index, lst_index)
    random_image= blur_img[r]
    plt.imshow(random_image)
    
plt.show()

blur_img = []
for i in images1:
  blur_img.append(cv2.GaussianBlur(i, (3, 3), 0))

plt.imshow(blur_img[3731])

<matplotlib.image.AxesImage at 0x7fe59f931d50>

# Visualize Images After Guassian Blurring

fig= plt.figure(figsize= (18, 40))
fig.suptitle('Random Samples From Each Class', fontsize=14, y=.92, horizontalalignment='center', weight='bold')

columns = 5
rows = 13

labelss = list(labels.Label.unique())

for i in range(12):
  lab = labelss[i]
  a = list(labels[labels['Label']==lab].index.values)
  fst_index = a[0]
  lst_index = a[-1]
  for j in range(1,6):
    fig.add_subplot(rows, columns, i*5+j)
    plt.axis('off')
    if j==1:
      plt.text(0.0, 0.5,lab, fontsize=15, wrap=True)
      continue
    r = randint(fst_index, lst_index)
    random_image= blur_img[r]
    plt.imshow(random_image)
    
plt.show()

Observations:

Back to Index

For observations, area between Y = 60 to 80 and X = 100 to 120 (area referring to tip of leaf is considered)

It is worth noting that there is some noise at the right hand side of the image.

1. Original Image: The leaf seems somwhat fuzzy with the noise.
2. Blurred 15*15: Fuzziness increases and it is difficult to separate leaf from noise.
3. Blurred 5 5 and 3 3: Edges appear distinct from noise. 3*3 filter will be chosen for further processing.
4. All these filters were tried during the course of this project lifecycle. However, the best results were produced with 3*3 filter.

Following Cell is for Illustration Purpose Only and Doesn't Impact Model / Data.

Back to Index

# Illustration purpose only:

# This is an attempt to reduce noise from the images to highlight leaves. 
# However, even after using this approach, model performance didn't improve.
# Therefore, it is now given for illustration purpose only. This can be improved in future iterations.


# Find the start and end indexes of each classification
lower_bound= (24, 50, 0)
upper_bound= (55, 255, 255)

fig= plt.figure(figsize=(12, 16))
fig.suptitle('Random Pre-Processed Image From Each Class', fontsize=14, y=.92, horizontalalignment='center', weight='bold')

columns = 5
rows = 13

labelss = list(labels.Label.unique())

for i in range(12):
  lab = labelss[i]
  a = list(labels[labels['Label']==lab].index.values)
  fst_index = a[0]
  lst_index = a[-1]
  for j in range(1,3):
    r = randint(fst_index, lst_index)
    random_image= images[r]
    img= random_image
    img= cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
    #img= cv2.resize(img, (150, 150))
    
    hsv_img= cv2.cvtColor(img, cv2.COLOR_RGB2HSV)
    mask = cv2.inRange(hsv_img, lower_bound, upper_bound)
    result = cv2.bitwise_and(img, img, mask=mask)

    #plt.imshow(result)
    fig.add_subplot(6, 4, i*2+1)
    plt.imshow(img)
    plt.axis('off')

    fig.add_subplot(6, 4, i*2+2)
    plt.imshow(result)
    plt.axis('off')
    
plt.show()

Following Cell is for Illustration Purpose Only and Doesn't Impact Model / Data.

Back to Index

# Illustration purpose only:


# Code mentioned in above cell works well to separate the leaf part of image from it's background.
# Therefore, this is an attempt to reduce noise from the images to highlight leaves. 
# However, even after using this approach, model performance didn't improve.
# Therefore, it is now given for illustration purpose only. This can be improved in future iterations.

images = np.load('images.npy')

# First convert the datatype of each RGB value in each pixel to float. Currently its a whole number 
# ranging between 0 - 255
images1 = images.astype('float32')

# For normalization, divide each value by 255
images1 /= 255

blur_img1 = []

for i in images1:
  hsv_img= cv2.cvtColor(i, cv2.COLOR_RGB2HSV)
  mask = cv2.inRange(hsv_img, (24, 50, 0), (55, 255, 255))
  result = cv2.bitwise_and(i, i, mask=mask)
  result= result.astype('float64')
  #blur_img1.append(result)
  blur_img1.append(cv2.GaussianBlur(result, (3, 3), 0))

Deliverable – 3:

Back to Index

Approach:

Back to Index

Make data compatible: (10 Marks)

1. Convert labels to one-hot-vectors.
2. Print the label for y_train[0].
3. Split the dataset into training, testing, and validation set. (Hint: First split images and labels into training and testing set with test_size = 0.3. Then further split test data into test and validation set with test_size = 0.5)
4. Check the shape of data, Reshape data into shapes compatible with Keras models if it’s not already. If it’s already in the compatible shape, then comment in the notebook that it’s already in compatible shape.

#Step1: Get all the labels
y = labels.Label

labels.nunique()

Label    12
dtype: int64

#Print first label in the array

print(labels.iloc[0])
y.iloc[0]

Label    Small-flowered Cranesbill
Name: 0, dtype: object

'Small-flowered Cranesbill'

Important Note:

Back to Index

Once labels in "Y-axis" are enumerated, the word names in classes will be replaced by numbers. It will make it difficult to understand and debug the model. Therefore, following code creates a dataframe containing the mapping between word and numeric values.

#Step2: Encode labels. Note: Step 3 is after a few cells.
label_encoder = LabelEncoder()

# First get all the labels in a variable called y for understanding purposes.
y = labels.Label
#print(y[1])

# Convert y from categorical (word) labels to numeric labels. This creates a numpy array
y1 = label_encoder.fit_transform(y)
#print(y1[1])
#print(type(y1))

# Convert above numpy array to Series
y_numeric = pd.Series(y1)

# Convert both series into a dictionary
df_y_dict = {'Label': y, 'Class_number': y_numeric}
print(type(df_y_dict))

# Finally convert dictionary into a dataframe
df_y = pd.DataFrame(df_y_dict)

# Drop duplicates from dataframe to retain 1:1 mapping between labels and their class number
df_y.drop_duplicates(inplace=True)

# Change y to reflect the numeric values. Optional step. 
# It has been carried out to maintain reference integrity of variable y
y = y1

df_y

<class 'dict'>

	Label	Class_number
0	Small-flowered Cranesbill	10
496	Fat Hen	5
971	Shepherds Purse	9
1202	Common wheat	4
1423	Common Chickweed	3
2034	Charlock	1
2424	Cleavers	2
2711	Scentless Mayweed	8
3227	Sugar beet	11
3612	Maize	7
3833	Black-grass	0
4096	Loose Silky-bent	6

Following Cell is for Illustration Purpose Only and Doesn't Impact Model / Data.

Back to Index

Approach:

Back to Index

As observed earlier, some classes are over-represented in data while some are under represented. Therefore, following code is an attempt to provide custom class weights to data so that low frequency classes can get higher weightage. Following is the approach:

1. Get % frequency for each class.
2. Divide 1 with frequency from point 1.
3. Take average (mean) of all the values from point 2.
4. Normalize values in point 2 above by dividing them by mean value obtained in point 3.

# Calculate Class Weights:

y_num = y_numeric.value_counts(normalize=True)
#print(y_num)

# This dictionary stores class and its weight. The same can be passed on while training model.
# Model paramter accepts dictionary input only for class_weight 
class_weight = {}

for i in range(12):
  class_weight[i] = (1/y_num[i])

sum = 0
for val in class_weight.values():
  sum += val
norm_factor= sum / len(class_weight)
#print(norm_factor)

for i in range(12):
  class_weight[i] = (1/y_num[i])/norm_factor

print(class_weight)

{0: 1.3017062904675036, 1: 0.877817318956291, 2: 1.1928528027629042, 3: 0.5603089269933772, 4: 1.5490893863934547, 5: 0.7207342197746389, 6: 0.5234690434143019, 7: 1.5490893863934547, 8: 0.6634665782809176, 9: 1.4820292397963355, 10: 0.6902192628890191, 11: 0.8892175438778014}

#Split train and test set. Validation split is not performed here.
#Validation split will be taken care while training the model.

X_train, X_test, y_train, y_test = train_test_split(blur_img, y, test_size=0.30, random_state=42)

X_train =  np.array(X_train)
X_test =  np.array(X_test)
y_train =  np.array(y_train)
y_test =  np.array(y_test)

#Check the shape of train and test sets

print(X_train.shape)
print(y_train.shape)
print(X_test.shape)
print(y_test.shape)

(3325, 128, 128, 3)
(3325,)
(1425, 128, 128, 3)
(1425,)

#Note that conversion of labels to one-hot-vectors is not yet completed.

#Print y_train[0]

print(y_train[0])
df_y[df_y['Class_number'] == y_train[0]]

2

	Label	Class_number
2424	Cleavers	2

Obserations:

Back to Index

The data is not yet ready for using in model. Need to work on Y-train and Y-test data further.

First re-split Y Test to form two new datasets: validation and Testing.

Use to_categorical function on Y train and Y validation data to convert them to categorical.

print("X_train shape:", X_train.shape)
print("Images in X_train:", X_train.shape[0])
print("Images in X_test:", X_test.shape[0])
print("Max value in X_train:", X_train.max())
print("Min value in X_train:", X_train.min())

X_train shape: (3325, 128, 128, 3)
Images in X_train: 3325
Images in X_test: 1425
Max value in X_train: 1.0
Min value in X_train: 0.0

#Step3: Convert labels to one-hot-vectors.
# Use to_categorical function on Y train and Y test data to convert them to categorical.

#from tensorflow.keras.utils import to_categorical

y_train = to_categorical(y_train, num_classes=12)
#y_test = to_categorical(y_test, num_classes=12)

print("Shape of y_train:", y_train.shape)
print("One value of y_train:", y_train[0])

Shape of y_train: (3325, 12)
One value of y_train: [0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0.]

#Print data shape again to confirm the conversion of Y data

print(X_train.shape)
print(y_train.shape)
print(X_test.shape)
print(y_test.shape)

(3325, 128, 128, 3)
(3325, 12)
(1425, 128, 128, 3)
(1425,)

#Split X_test further into testing and validation sets.

X_test_t, X_validation, y_test_t, y_validation = train_test_split(X_test, y_test, test_size=0.50, random_state=42)

y_validation = to_categorical(y_validation, num_classes=12)

#Note: Y test need not undergo to_categorical transformation.

print(X_test_t.shape)
print(y_test_t.shape)
print(X_validation.shape)
print(y_validation.shape)

(712, 128, 128, 3)
(712,)
(713, 128, 128, 3)
(713, 12)

Observations:

Back to Index

1. 70% training data and 30% testing data.

2. Testing data is further split into 50% test and 50% validation sets.

3. Y-Data (train and validation) was converted to categorical, and now we are ready to train and test the model.

4. Similar number of records in training testing and validation sets.

Deliverable – 4:

Back to Index

Approach:

Back to Index

Building CNN (15 Marks)

1. Define layers.
2. Set optimizer and loss function. (Use Adam optimizer and categorical crossentropy.)

Note: I have tried three callbacks, "EarlyStopping", "ReduceLROnPlateau", and "ModelCheckpoint" during the model building process. However, "ModelCheckpoint" performed better, and therefore is used for final model selection process. Other callback methods are given here for illustration purposes only.

# Initialize the model
model = models.Sequential()

# Add a Convolutional Layer with 32 filters of size 3X3 and activation function as 'relu' 
model.add(Conv2D(filters=32, kernel_size=(3,3), padding='same', activation="relu", input_shape=(128, 128, 3)))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.1))

# Add a Convolutional Layer with 32 filters of size 3X3 and activation function as 'relu' 
model.add(Conv2D(filters=32, kernel_size=(3,3), padding='same', activation="relu", input_shape=(128, 128, 3)))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.2))

# Add a Convolutional Layer with 64 filters of size 3X3 and activation function as 'relu' 
model.add(Conv2D(filters=64, kernel_size=(3,3), padding='same', activation="relu", input_shape=(128, 128, 3)))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.2))

# Add a Convolutional Layer with 64 filters of size 3X3 and activation function as 'relu' 
model.add(Conv2D(filters=64, kernel_size=(3,3), padding='same', activation="relu", input_shape=(128, 128, 3)))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.2))

# Add a Convolutional Layer with 64 filters of size 3X3 and activation function as 'relu' 
model.add(Conv2D(filters=64, kernel_size=(3,3), padding='same', activation="relu", input_shape=(128, 128, 3)))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.2))

# Add a Convolutional Layer with 128 filters of size 3X3 and activation function as 'relu' 
model.add(Conv2D(filters=128, kernel_size=3, padding='same', activation="relu", input_shape=(128, 128, 3)))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.3))

# Add a Convolutional Layer with 128 filters of size 3X3 and activation function as 'relu' 
model.add(Conv2D(filters=128, kernel_size=3, padding='same', activation="relu", input_shape=(128, 128, 3)))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.1))

# Flatten the layer
model.add(Flatten())
model.add(Dropout(0.1))

# Add Fully Connected Layer with 512 units and activation function as 'relu'
model.add(Dense(512, activation="relu"))
model.add(Dropout(0.4))

# Add Fully Connected Layer with 256 units and activation function as 'relu'
model.add(Dense(256, activation="relu"))
model.add(Dropout(0.4))

#model.add(Dense(32, activation="relu"))
#model.add(Dropout(0.1))

#Add Fully Connected Layer with 10 units and activation function as 'softmax'
model.add(Dense(12, activation="softmax"))

#Print model summary

model.summary()

Model: "sequential_2"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_13 (Conv2D)           (None, 128, 128, 32)      896       
_________________________________________________________________
max_pooling2d_13 (MaxPooling (None, 64, 64, 32)        0         
_________________________________________________________________
dropout_19 (Dropout)         (None, 64, 64, 32)        0         
_________________________________________________________________
conv2d_14 (Conv2D)           (None, 64, 64, 32)        9248      
_________________________________________________________________
max_pooling2d_14 (MaxPooling (None, 32, 32, 32)        0         
_________________________________________________________________
dropout_20 (Dropout)         (None, 32, 32, 32)        0         
_________________________________________________________________
conv2d_15 (Conv2D)           (None, 32, 32, 64)        18496     
_________________________________________________________________
max_pooling2d_15 (MaxPooling (None, 16, 16, 64)        0         
_________________________________________________________________
dropout_21 (Dropout)         (None, 16, 16, 64)        0         
_________________________________________________________________
conv2d_16 (Conv2D)           (None, 16, 16, 64)        36928     
_________________________________________________________________
max_pooling2d_16 (MaxPooling (None, 8, 8, 64)          0         
_________________________________________________________________
dropout_22 (Dropout)         (None, 8, 8, 64)          0         
_________________________________________________________________
conv2d_17 (Conv2D)           (None, 8, 8, 128)         73856     
_________________________________________________________________
max_pooling2d_17 (MaxPooling (None, 4, 4, 128)         0         
_________________________________________________________________
dropout_23 (Dropout)         (None, 4, 4, 128)         0         
_________________________________________________________________
conv2d_18 (Conv2D)           (None, 4, 4, 128)         147584    
_________________________________________________________________
max_pooling2d_18 (MaxPooling (None, 2, 2, 128)         0         
_________________________________________________________________
dropout_24 (Dropout)         (None, 2, 2, 128)         0         
_________________________________________________________________
flatten_2 (Flatten)          (None, 512)               0         
_________________________________________________________________
dropout_25 (Dropout)         (None, 512)               0         
_________________________________________________________________
dense_6 (Dense)              (None, 512)               262656    
_________________________________________________________________
dropout_26 (Dropout)         (None, 512)               0         
_________________________________________________________________
dense_7 (Dense)              (None, 256)               131328    
_________________________________________________________________
dropout_27 (Dropout)         (None, 256)               0         
_________________________________________________________________
dense_8 (Dense)              (None, 12)                3084      
=================================================================
Total params: 684,076
Trainable params: 684,076
Non-trainable params: 0
_________________________________________________________________

# Set Optimizer to Adam And loss to "categorical_crossentropy"
opt= keras.optimizers.Adam(lr=0.0005, amsgrad=True)

# Compile the model
model.compile(loss="categorical_crossentropy", metrics=["accuracy"], optimizer=opt)

# Use one of the following callbacks.

# EarlyStopping: Didn't work the best, unable to set an optimal value for patience. Sometimes it goes low and sometimes very high
callback = tensorflow.keras.callbacks.EarlyStopping(monitor='val_accuracy', patience=2, min_delta=0.01)

# ReduceLROnPlateau: Didn't work the best.
reduce_lr = tensorflow.keras.callbacks.ReduceLROnPlateau(monitor='val_loss', factor=0.2,
                              patience=5, min_lr=0.001)

# This worked the best since number of epochs is not large.
best_cb= tensorflow.keras.callbacks.ModelCheckpoint('model_best.h5', 
                                         monitor='val_loss', 
                                         verbose=1, 
                                         save_best_only=True, 
                                         save_weights_only=False, 
                                         mode='auto')

# Fit the model
history = model.fit(x=X_train
                    , y=y_train, steps_per_epoch= 190
                    , epochs=75
                    #, class_weight = class_weight #Didn't use the class weights calculated above, it impacts model performance.
                    , validation_data=(X_validation, y_validation)
                    , callbacks=[best_cb])

Epoch 1/75
190/190 [==============================] - 4s 16ms/step - loss: 2.4508 - accuracy: 0.1295 - val_loss: 2.4299 - val_accuracy: 0.1346

Epoch 00001: val_loss improved from inf to 2.42993, saving model to model_best.h5
Epoch 2/75
190/190 [==============================] - 3s 14ms/step - loss: 2.4255 - accuracy: 0.1386 - val_loss: 2.4047 - val_accuracy: 0.1346

Epoch 00002: val_loss improved from 2.42993 to 2.40473, saving model to model_best.h5
Epoch 3/75
190/190 [==============================] - 3s 14ms/step - loss: 2.3564 - accuracy: 0.1811 - val_loss: 1.9727 - val_accuracy: 0.3156

Epoch 00003: val_loss improved from 2.40473 to 1.97274, saving model to model_best.h5
Epoch 4/75
190/190 [==============================] - 3s 14ms/step - loss: 1.9011 - accuracy: 0.3337 - val_loss: 1.7181 - val_accuracy: 0.4208

Epoch 00004: val_loss improved from 1.97274 to 1.71808, saving model to model_best.h5
Epoch 5/75
190/190 [==============================] - 3s 13ms/step - loss: 1.6964 - accuracy: 0.3982 - val_loss: 1.4650 - val_accuracy: 0.4755

Epoch 00005: val_loss improved from 1.71808 to 1.46499, saving model to model_best.h5
Epoch 6/75
190/190 [==============================] - 3s 14ms/step - loss: 1.4759 - accuracy: 0.4704 - val_loss: 1.3240 - val_accuracy: 0.5386

Epoch 00006: val_loss improved from 1.46499 to 1.32399, saving model to model_best.h5
Epoch 7/75
190/190 [==============================] - 3s 14ms/step - loss: 1.3153 - accuracy: 0.5141 - val_loss: 1.1537 - val_accuracy: 0.5877

Epoch 00007: val_loss improved from 1.32399 to 1.15370, saving model to model_best.h5
Epoch 8/75
190/190 [==============================] - 3s 13ms/step - loss: 1.1939 - accuracy: 0.5733 - val_loss: 1.2339 - val_accuracy: 0.5540

Epoch 00008: val_loss did not improve from 1.15370
Epoch 9/75
190/190 [==============================] - 3s 14ms/step - loss: 1.0405 - accuracy: 0.6249 - val_loss: 0.9442 - val_accuracy: 0.6522

Epoch 00009: val_loss improved from 1.15370 to 0.94417, saving model to model_best.h5
Epoch 10/75
190/190 [==============================] - 3s 13ms/step - loss: 0.9736 - accuracy: 0.6681 - val_loss: 0.8750 - val_accuracy: 0.6942

Epoch 00010: val_loss improved from 0.94417 to 0.87501, saving model to model_best.h5
Epoch 11/75
190/190 [==============================] - 3s 14ms/step - loss: 0.9379 - accuracy: 0.6792 - val_loss: 0.8977 - val_accuracy: 0.6957

Epoch 00011: val_loss did not improve from 0.87501
Epoch 12/75
190/190 [==============================] - 3s 14ms/step - loss: 0.8432 - accuracy: 0.7055 - val_loss: 0.8019 - val_accuracy: 0.7433

Epoch 00012: val_loss improved from 0.87501 to 0.80188, saving model to model_best.h5
Epoch 13/75
190/190 [==============================] - 3s 14ms/step - loss: 0.8056 - accuracy: 0.7226 - val_loss: 0.8467 - val_accuracy: 0.7335

Epoch 00013: val_loss did not improve from 0.80188
Epoch 14/75
190/190 [==============================] - 3s 14ms/step - loss: 0.7140 - accuracy: 0.7526 - val_loss: 0.8410 - val_accuracy: 0.7237

Epoch 00014: val_loss did not improve from 0.80188
Epoch 15/75
190/190 [==============================] - 3s 13ms/step - loss: 0.6269 - accuracy: 0.7820 - val_loss: 0.7824 - val_accuracy: 0.7504

Epoch 00015: val_loss improved from 0.80188 to 0.78244, saving model to model_best.h5
Epoch 16/75
190/190 [==============================] - 3s 14ms/step - loss: 0.6005 - accuracy: 0.7840 - val_loss: 0.6537 - val_accuracy: 0.7700

Epoch 00016: val_loss improved from 0.78244 to 0.65368, saving model to model_best.h5
Epoch 17/75
190/190 [==============================] - 3s 14ms/step - loss: 0.5738 - accuracy: 0.7977 - val_loss: 0.6714 - val_accuracy: 0.7784

Epoch 00017: val_loss did not improve from 0.65368
Epoch 18/75
190/190 [==============================] - 3s 13ms/step - loss: 0.5366 - accuracy: 0.8038 - val_loss: 0.7039 - val_accuracy: 0.7770

Epoch 00018: val_loss did not improve from 0.65368
Epoch 19/75
190/190 [==============================] - 3s 14ms/step - loss: 0.4928 - accuracy: 0.8306 - val_loss: 0.6117 - val_accuracy: 0.8163

Epoch 00019: val_loss improved from 0.65368 to 0.61174, saving model to model_best.h5
Epoch 20/75
190/190 [==============================] - 3s 14ms/step - loss: 0.4809 - accuracy: 0.8307 - val_loss: 0.6011 - val_accuracy: 0.8050

Epoch 00020: val_loss improved from 0.61174 to 0.60115, saving model to model_best.h5
Epoch 21/75
190/190 [==============================] - 3s 13ms/step - loss: 0.3977 - accuracy: 0.8498 - val_loss: 0.6067 - val_accuracy: 0.7924

Epoch 00021: val_loss did not improve from 0.60115
Epoch 22/75
190/190 [==============================] - 3s 14ms/step - loss: 0.4298 - accuracy: 0.8490 - val_loss: 0.5162 - val_accuracy: 0.8289

Epoch 00022: val_loss improved from 0.60115 to 0.51617, saving model to model_best.h5
Epoch 23/75
190/190 [==============================] - 3s 14ms/step - loss: 0.3551 - accuracy: 0.8802 - val_loss: 0.5233 - val_accuracy: 0.8261

Epoch 00023: val_loss did not improve from 0.51617
Epoch 24/75
190/190 [==============================] - 3s 14ms/step - loss: 0.3163 - accuracy: 0.8864 - val_loss: 0.6310 - val_accuracy: 0.8050

Epoch 00024: val_loss did not improve from 0.51617
Epoch 25/75
190/190 [==============================] - 3s 13ms/step - loss: 0.3634 - accuracy: 0.8630 - val_loss: 0.5497 - val_accuracy: 0.8303

Epoch 00025: val_loss did not improve from 0.51617
Epoch 26/75
190/190 [==============================] - 3s 13ms/step - loss: 0.3682 - accuracy: 0.8685 - val_loss: 0.5054 - val_accuracy: 0.8247

Epoch 00026: val_loss improved from 0.51617 to 0.50538, saving model to model_best.h5
Epoch 27/75
190/190 [==============================] - 3s 13ms/step - loss: 0.3187 - accuracy: 0.8854 - val_loss: 0.4800 - val_accuracy: 0.8415

Epoch 00027: val_loss improved from 0.50538 to 0.47999, saving model to model_best.h5
Epoch 28/75
190/190 [==============================] - 3s 14ms/step - loss: 0.3177 - accuracy: 0.8825 - val_loss: 0.4794 - val_accuracy: 0.8527

Epoch 00028: val_loss improved from 0.47999 to 0.47937, saving model to model_best.h5
Epoch 29/75
190/190 [==============================] - 3s 13ms/step - loss: 0.2809 - accuracy: 0.8911 - val_loss: 0.4715 - val_accuracy: 0.8387

Epoch 00029: val_loss improved from 0.47937 to 0.47147, saving model to model_best.h5
Epoch 30/75
190/190 [==============================] - 3s 14ms/step - loss: 0.2910 - accuracy: 0.8924 - val_loss: 0.5241 - val_accuracy: 0.8163

Epoch 00030: val_loss did not improve from 0.47147
Epoch 31/75
190/190 [==============================] - 3s 14ms/step - loss: 0.2850 - accuracy: 0.8980 - val_loss: 0.4787 - val_accuracy: 0.8387

Epoch 00031: val_loss did not improve from 0.47147
Epoch 32/75
190/190 [==============================] - 3s 14ms/step - loss: 0.2684 - accuracy: 0.9021 - val_loss: 0.4812 - val_accuracy: 0.8457

Epoch 00032: val_loss did not improve from 0.47147
Epoch 33/75
190/190 [==============================] - 3s 14ms/step - loss: 0.2861 - accuracy: 0.8973 - val_loss: 0.5178 - val_accuracy: 0.8429

Epoch 00033: val_loss did not improve from 0.47147
Epoch 34/75
190/190 [==============================] - 3s 14ms/step - loss: 0.2605 - accuracy: 0.9013 - val_loss: 0.4754 - val_accuracy: 0.8499

Epoch 00034: val_loss did not improve from 0.47147
Epoch 35/75
190/190 [==============================] - 3s 14ms/step - loss: 0.2388 - accuracy: 0.9120 - val_loss: 0.5666 - val_accuracy: 0.8177

Epoch 00035: val_loss did not improve from 0.47147
Epoch 36/75
190/190 [==============================] - 3s 13ms/step - loss: 0.2595 - accuracy: 0.9083 - val_loss: 0.4383 - val_accuracy: 0.8583

Epoch 00036: val_loss improved from 0.47147 to 0.43831, saving model to model_best.h5
Epoch 37/75
190/190 [==============================] - 3s 14ms/step - loss: 0.2325 - accuracy: 0.9142 - val_loss: 0.4392 - val_accuracy: 0.8527

Epoch 00037: val_loss did not improve from 0.43831
Epoch 38/75
190/190 [==============================] - 3s 14ms/step - loss: 0.2252 - accuracy: 0.9140 - val_loss: 0.5190 - val_accuracy: 0.8331

Epoch 00038: val_loss did not improve from 0.43831
Epoch 39/75
190/190 [==============================] - 3s 14ms/step - loss: 0.2272 - accuracy: 0.9131 - val_loss: 0.5222 - val_accuracy: 0.8387

Epoch 00039: val_loss did not improve from 0.43831
Epoch 40/75
190/190 [==============================] - 3s 14ms/step - loss: 0.2547 - accuracy: 0.9088 - val_loss: 0.5225 - val_accuracy: 0.8527

Epoch 00040: val_loss did not improve from 0.43831
Epoch 41/75
190/190 [==============================] - 3s 14ms/step - loss: 0.2135 - accuracy: 0.9178 - val_loss: 0.4247 - val_accuracy: 0.8738

Epoch 00041: val_loss improved from 0.43831 to 0.42473, saving model to model_best.h5
Epoch 42/75
190/190 [==============================] - 3s 14ms/step - loss: 0.1854 - accuracy: 0.9343 - val_loss: 0.5125 - val_accuracy: 0.8499

Epoch 00042: val_loss did not improve from 0.42473
Epoch 43/75
190/190 [==============================] - 3s 14ms/step - loss: 0.2291 - accuracy: 0.9207 - val_loss: 0.4480 - val_accuracy: 0.8583

Epoch 00043: val_loss did not improve from 0.42473
Epoch 44/75
190/190 [==============================] - 3s 14ms/step - loss: 0.1788 - accuracy: 0.9348 - val_loss: 0.5047 - val_accuracy: 0.8485

Epoch 00044: val_loss did not improve from 0.42473
Epoch 45/75
190/190 [==============================] - 3s 14ms/step - loss: 0.2189 - accuracy: 0.9147 - val_loss: 0.4678 - val_accuracy: 0.8640

Epoch 00045: val_loss did not improve from 0.42473
Epoch 46/75
190/190 [==============================] - 3s 14ms/step - loss: 0.1680 - accuracy: 0.9310 - val_loss: 0.4614 - val_accuracy: 0.8626

Epoch 00046: val_loss did not improve from 0.42473
Epoch 47/75
190/190 [==============================] - 3s 14ms/step - loss: 0.1780 - accuracy: 0.9309 - val_loss: 0.4631 - val_accuracy: 0.8555

Epoch 00047: val_loss did not improve from 0.42473
Epoch 48/75
190/190 [==============================] - 3s 14ms/step - loss: 0.1698 - accuracy: 0.9336 - val_loss: 0.5173 - val_accuracy: 0.8485

Epoch 00048: val_loss did not improve from 0.42473
Epoch 49/75
190/190 [==============================] - 3s 14ms/step - loss: 0.1609 - accuracy: 0.9371 - val_loss: 0.4987 - val_accuracy: 0.8612

Epoch 00049: val_loss did not improve from 0.42473
Epoch 50/75
190/190 [==============================] - 3s 13ms/step - loss: 0.2181 - accuracy: 0.9177 - val_loss: 0.5381 - val_accuracy: 0.8527

Epoch 00050: val_loss did not improve from 0.42473
Epoch 51/75
190/190 [==============================] - 3s 14ms/step - loss: 0.2034 - accuracy: 0.9187 - val_loss: 0.4832 - val_accuracy: 0.8485

Epoch 00051: val_loss did not improve from 0.42473
Epoch 52/75
190/190 [==============================] - 3s 14ms/step - loss: 0.2151 - accuracy: 0.9247 - val_loss: 0.4911 - val_accuracy: 0.8640

Epoch 00052: val_loss did not improve from 0.42473
Epoch 53/75
190/190 [==============================] - 3s 14ms/step - loss: 0.1672 - accuracy: 0.9339 - val_loss: 0.5260 - val_accuracy: 0.8401

Epoch 00053: val_loss did not improve from 0.42473
Epoch 54/75
190/190 [==============================] - 3s 14ms/step - loss: 0.1656 - accuracy: 0.9393 - val_loss: 0.5106 - val_accuracy: 0.8541

Epoch 00054: val_loss did not improve from 0.42473
Epoch 55/75
190/190 [==============================] - 3s 14ms/step - loss: 0.1949 - accuracy: 0.9368 - val_loss: 0.4840 - val_accuracy: 0.8612

Epoch 00055: val_loss did not improve from 0.42473
Epoch 56/75
190/190 [==============================] - 3s 14ms/step - loss: 0.1520 - accuracy: 0.9442 - val_loss: 0.4914 - val_accuracy: 0.8626

Epoch 00056: val_loss did not improve from 0.42473
Epoch 57/75
190/190 [==============================] - 3s 14ms/step - loss: 0.1531 - accuracy: 0.9388 - val_loss: 0.4422 - val_accuracy: 0.8710

Epoch 00057: val_loss did not improve from 0.42473
Epoch 58/75
190/190 [==============================] - 3s 14ms/step - loss: 0.1294 - accuracy: 0.9481 - val_loss: 0.4373 - val_accuracy: 0.8668

Epoch 00058: val_loss did not improve from 0.42473
Epoch 59/75
190/190 [==============================] - 3s 14ms/step - loss: 0.1255 - accuracy: 0.9605 - val_loss: 0.4707 - val_accuracy: 0.8682

Epoch 00059: val_loss did not improve from 0.42473
Epoch 60/75
190/190 [==============================] - 3s 14ms/step - loss: 0.1656 - accuracy: 0.9414 - val_loss: 0.5315 - val_accuracy: 0.8541

Epoch 00060: val_loss did not improve from 0.42473
Epoch 61/75
190/190 [==============================] - 3s 14ms/step - loss: 0.1856 - accuracy: 0.9396 - val_loss: 0.5056 - val_accuracy: 0.8555

Epoch 00061: val_loss did not improve from 0.42473
Epoch 62/75
190/190 [==============================] - 3s 13ms/step - loss: 0.1230 - accuracy: 0.9555 - val_loss: 0.4248 - val_accuracy: 0.8682

Epoch 00062: val_loss did not improve from 0.42473
Epoch 63/75
190/190 [==============================] - 3s 14ms/step - loss: 0.1415 - accuracy: 0.9458 - val_loss: 0.4418 - val_accuracy: 0.8752

Epoch 00063: val_loss did not improve from 0.42473
Epoch 64/75
190/190 [==============================] - 3s 14ms/step - loss: 0.1370 - accuracy: 0.9503 - val_loss: 0.4520 - val_accuracy: 0.8668

Epoch 00064: val_loss did not improve from 0.42473
Epoch 65/75
190/190 [==============================] - 3s 14ms/step - loss: 0.1191 - accuracy: 0.9544 - val_loss: 0.4703 - val_accuracy: 0.8654

Epoch 00065: val_loss did not improve from 0.42473
Epoch 66/75
190/190 [==============================] - 3s 14ms/step - loss: 0.1315 - accuracy: 0.9483 - val_loss: 0.4569 - val_accuracy: 0.8682

Epoch 00066: val_loss did not improve from 0.42473
Epoch 67/75
190/190 [==============================] - 3s 14ms/step - loss: 0.1271 - accuracy: 0.9501 - val_loss: 0.4758 - val_accuracy: 0.8583

Epoch 00067: val_loss did not improve from 0.42473
Epoch 68/75
190/190 [==============================] - 3s 14ms/step - loss: 0.1125 - accuracy: 0.9600 - val_loss: 0.4839 - val_accuracy: 0.8668

Epoch 00068: val_loss did not improve from 0.42473
Epoch 69/75
190/190 [==============================] - 3s 14ms/step - loss: 0.1203 - accuracy: 0.9557 - val_loss: 0.4735 - val_accuracy: 0.8738

Epoch 00069: val_loss did not improve from 0.42473
Epoch 70/75
190/190 [==============================] - 3s 14ms/step - loss: 0.1401 - accuracy: 0.9421 - val_loss: 0.4906 - val_accuracy: 0.8710

Epoch 00070: val_loss did not improve from 0.42473
Epoch 71/75
190/190 [==============================] - 3s 14ms/step - loss: 0.1324 - accuracy: 0.9485 - val_loss: 0.4708 - val_accuracy: 0.8668

Epoch 00071: val_loss did not improve from 0.42473
Epoch 72/75
190/190 [==============================] - 3s 14ms/step - loss: 0.1124 - accuracy: 0.9597 - val_loss: 0.4747 - val_accuracy: 0.8696

Epoch 00072: val_loss did not improve from 0.42473
Epoch 73/75
190/190 [==============================] - 3s 14ms/step - loss: 0.1297 - accuracy: 0.9573 - val_loss: 0.4570 - val_accuracy: 0.8640

Epoch 00073: val_loss did not improve from 0.42473
Epoch 74/75
  1/190 [..............................] - ETA: 2s - loss: 0.1602 - accuracy: 0.9444WARNING:tensorflow:Your input ran out of data; interrupting training. Make sure that your dataset or generator can generate at least `steps_per_epoch * epochs` batches (in this case, 14250 batches). You may need to use the repeat() function when building your dataset.
190/190 [==============================] - 0s 1ms/step - loss: 0.2427 - accuracy: 0.8827 - val_loss: 0.4568 - val_accuracy: 0.8668

Epoch 00074: val_loss did not improve from 0.42473

#history.history

acc = history.history['accuracy']
val_acc = history.history['val_accuracy']
loss = history.history['loss']
val_loss = history.history['val_loss']
epochs = range(1, len(acc) + 1)
plt.figure()
plt.plot(epochs, acc, 'bo', label='Training acc')
plt.plot(epochs, val_acc, 'b', label='Validation acc')
plt.title('Training and validation accuracy')
plt.legend()
plt.figure()
plt.plot(epochs, loss, 'bo', label='Training loss')
plt.plot(epochs, val_loss, 'b', label='Validation loss')
plt.title('Training and validation loss')
plt.legend()
plt.show()

Observation:

Back to Index

Both validation loss and validation accuracy stabilize around 25th epoch.

Deliverable – 5:

Back to Index

Approach:

Back to Index

Fit and evaluate model and print confusion matrix.

# Following code gives the best model that will be used for testing

model_b = models.load_model('model_best.h5')

# Prediction

y_pred = model_b.predict(X_test_t)
print(y_pred[1])

[1.3251375e-07 1.0520875e-04 3.4479979e-02 2.3869261e-04 2.3065550e-04
 7.9145888e-04 4.9828859e-08 3.9085074e-04 9.5196426e-01 1.0827353e-02
 2.1362746e-06 9.6926268e-04]

Observation:

Back to Index

9.5842135e-01 = 0.95842135. This explains that the image belongs to class 9.

# The above code gives the probability of an image to be classified as a class.
# Now Convert those y-prediction probabilities from encdoed to a numeric class using argmax

from sklearn.metrics import accuracy_score
rounded_labels=np.argmax(y_pred, axis=1)
rounded_labels[1]

8

from sklearn.metrics import classification_report as cr
from sklearn.metrics import confusion_matrix as cm

print('Classification Report')
print(cr(y_test_t, rounded_labels))

Classification Report
              precision    recall  f1-score   support

           0       0.65      0.32      0.43        41
           1       0.94      0.84      0.88        55
           2       0.75      0.92      0.83        49
           3       0.95      0.95      0.95        88
           4       0.77      0.82      0.79        33
           5       0.97      0.88      0.92        74
           6       0.77      0.94      0.84       101
           7       0.90      0.93      0.92        30
           8       0.76      0.97      0.85        74
           9       0.89      0.65      0.75        48
          10       0.97      0.90      0.94        72
          11       0.98      0.85      0.91        47

    accuracy                           0.86       712
   macro avg       0.86      0.83      0.83       712
weighted avg       0.86      0.86      0.85       712

# Print Label to class number mapping dataframe for easier understanding of the above and below matrices.

df_y

	Label	Class_number
0	Small-flowered Cranesbill	10
496	Fat Hen	5
971	Shepherds Purse	9
1202	Common wheat	4
1423	Common Chickweed	3
2034	Charlock	1
2424	Cleavers	2
2711	Scentless Mayweed	8
3227	Sugar beet	11
3612	Maize	7
3833	Black-grass	0
4096	Loose Silky-bent	6

#Y_pred = model.predict_generator(X_test, 128 // batch_size+1)
#y_pred = np.argmax(Y_pred, axis=1)
print('Confusion Matrix')
matrix = cm(y_test_t, rounded_labels)
print(matrix)

Confusion Matrix
[[13  0  0  0  4  0 24  0  0  0  0  0]
 [ 0 46  6  0  0  0  0  0  1  2  0  0]
 [ 0  1 45  1  1  0  0  0  1  0  0  0]
 [ 0  0  0 84  0  0  0  0  3  0  1  0]
 [ 2  0  0  0 27  1  2  1  0  0  0  0]
 [ 0  1  4  0  0 65  3  0  0  1  0  0]
 [ 5  0  0  0  1  0 95  0  0  0  0  0]
 [ 0  0  1  0  1  0  0 28  0  0  0  0]
 [ 0  0  1  0  0  0  0  1 72  0  0  0]
 [ 0  0  0  2  1  1  0  0 12 31  0  1]
 [ 0  1  2  1  0  0  0  0  2  1 65  0]
 [ 0  0  1  0  0  0  0  1  4  0  1 40]]

df_cm = pd.DataFrame(matrix, index = [i for i in range(12)],
                     columns = [i for i in range(12)])
plt.figure(figsize = (10,7))
sns.heatmap(df_cm, annot=True, fmt='d')

<matplotlib.axes._subplots.AxesSubplot at 0x7fe59d1b1a10>

Observations:

Back to Index

1. Most of the classes have good scores for precision and recall (both more than 0.8). Therefore, their F1 scores are also good.
2. Model is mis-classifying "Black-Grass" (Class 0) as "Loose Silky-Bent". There can be multiple causes for this mis-classification:
   • "Black-Grass" is has less data (263 records including 41 test cases) in the dataset than "Loose Silky-Bent" (654 records including 101 test cases).
   • If we observe the data visualization for these two classes, it is difficult even for human eye to classify between the two.
   • Only a small portion of the image contains data, while a large portion comprises of image background.
   • Different angles from which images are taken.
   • Even some "Loose Silky-Bent" are mis-classified as "Black-Grass"
3. There are 12 cases from Class 9 which are mis-classified as class 8, but no classes from class 8 are mis-classified as class 9.

Overall, the model performs well to label most of the images, except for "Black-grass". In the next iteration of the model, some more images from this class can be tested for model improvement.

Note: The numbers in confusion matrix may change after each run of the model.

Deliverable – 6:

Back to Index

Approach:

Back to Index

Visualize predictions for x_test[2], x_test[3], x_test[33], x_test[36], x_test[59].

print("Following is the label from Y-Test")
df_y[df_y['Class_number'] == y_test_t[2]]

Following is the label from Y-Test

	Label	Class_number
4096	Loose Silky-bent	6

print("Following is the label from Y-Predict")
df_y[df_y['Class_number'] == rounded_labels[2]]

Following is the label from Y-Predict

	Label	Class_number
4096	Loose Silky-bent	6

print("Following is the corresponding image")
plt.imshow(X_test_t[2])

Following is the corresponding image

<matplotlib.image.AxesImage at 0x7fe550044d90>

print("Following is the label from Y-Test")
df_y[df_y['Class_number'] == y_test_t[3]]

Following is the label from Y-Test

	Label	Class_number
1423	Common Chickweed	3

print("Following is the label from Y-Predict")
df_y[df_y['Class_number'] == rounded_labels[3]]

Following is the label from Y-Predict

	Label	Class_number
1423	Common Chickweed	3

print("Following is the corresponding image")
plt.imshow(X_test_t[3])

Following is the corresponding image

<matplotlib.image.AxesImage at 0x7fe54864f590>

print("Following is the label from Y-Test")
df_y[df_y['Class_number'] == y_test_t[33]]

Following is the label from Y-Test

	Label	Class_number
1202	Common wheat	4

print("Following is the label from Y-Predict")
df_y[df_y['Class_number'] == rounded_labels[33]]

Following is the label from Y-Predict

	Label	Class_number
3833	Black-grass	0

print("Following is the corresponding image")
plt.imshow(X_test_t[33])

Following is the corresponding image

<matplotlib.image.AxesImage at 0x7fe548198c50>

print("Following is the label from Y-Test")
df_y[df_y['Class_number'] == y_test_t[36]]

Following is the label from Y-Test

	Label	Class_number
1202	Common wheat	4

print("Following is the label from Y-Predict")
df_y[df_y['Class_number'] == rounded_labels[36]]

Following is the label from Y-Predict

	Label	Class_number
1202	Common wheat	4

print("Following is the corresponding image")
plt.imshow(X_test_t[36])

Following is the corresponding image

<matplotlib.image.AxesImage at 0x7fe548112490>

print("Following is the label from Y-Test")
df_y[df_y['Class_number'] == y_test_t[59]]

Following is the label from Y-Test

	Label	Class_number
0	Small-flowered Cranesbill	10

print("Following is the label from Y-Predict")
df_y[df_y['Class_number'] == rounded_labels[59]]

Following is the label from Y-Predict

	Label	Class_number
0	Small-flowered Cranesbill	10

print("Following is the corresponding image")
plt.imshow(X_test_t[59])

Following is the corresponding image

<matplotlib.image.AxesImage at 0x7fe5480fda50>

From the samples in scope of this deliverable, one was misclassified, while others were properly labelled. The results may vary after each run of the model.